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IMAGE PROCESSING APPARATUS AND RECORDING MEDIUM, AND IMAGE 

PROCESSING APPARATUS 

BACKGROUND OF THE INVENTION 
5 The present invention relates to an image processingmethod, 

a recording medium, and an image processing apparatus for 
jK* smoothing digital data for an input image and for removing 

noise . 

For an image input device such as a digital still camera 
10 (hereinafter referred to as digital cameras) or a scanner 
that use image input unit, including a CCD, the image input 
unit sometimes outputs not only an electric signal corresponding 
to the intensity of the light received, but also an unwanted 

•SI electric signal. This unwanted electrical signal is included 

111 

15 as noise in the digital data constituting an image. The 
occurrence of this noise tends to be especially remarkable 
in a low intensity luminous portion, i.e., a dark image portion . 

Generally, in order to remove such noise included in an 
image, such a method is employed that smoothes all the digital 
20 data constituting an image, or a method is employed that uses 
edge information to remove the included noise. 

For the removal of noise while maintaining the edge 
information, a method (a) for smoothing image data or for 
emphasizing contours in accordance with the slopes of the 
25 edges of an image; and a method (b) for detecting the direction 
of edges and changing smoothing ranges are presently available . 
Herein the slopes of the edges are the slopes of the luminance 
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of the pixels that constitute an image. 

However, according to the method (a) for smoothing the 
image data or for emphasizing the contours in accordance with 
the slopes of the edges, noise present in a flat portion wherein 
the amount of noise is comparatively less is also emphasized, 
and the image quality is deteriorated. On the other hand, 
according to the method (b) for detecting the direction of 
an edge so as to change a smoothing range, conditional branching 
is employed to detect the direction of the edge. Therefore, 
when angles used for detecting the direction of the edge are 
set in detail in order to increase the accuracy of the detection 
angles, the number of conditions imposed for the performance 
of the branching is increased. Therefore, it is difficult 
to perform fast processing such as pipeline processing. In 
addition, when the angles for the detection of the direction 
of an edge are roughly set in order to increase the processing 
speed, errors occur more frequently during the determination 
processing performed for conditional branching, and the edge 
tends to be blurred. 

Further, the methods (a) and (b) are performed as 
independent functions, and there is no method for jointly 
implementing the functions of the two methods. 

SUMMARY OF THE INVENTION 
It is, therefore, one objective of the present invention 
to provide an image processing method and a recording medium 
therefor, and an image processing apparatus for quickly removing 



noise without blurring the edges of images or emphasizing 
the noise present in flat portions. 

(1) An image processing method for smoothing digital data 
for an input image and for removing noise included in the 
digital data, the method comprising: 

an edge information calculation step of extracting an 
edge of the image from the digital data, and calculating edge 
information including a grade and a direction of a slope of 
the edge; 

a filter information selection step of selecting filter 
information that is set in advance based on the edge information 
obtained at the edge information calculation step; and 

a processing step of smoothing the digital data based 
on the filter information selected at the filter information 
selection step. 

(2) The image processing method according to (1), wherein, 
at the filter information selection step, when the grade of 
the slope is small, filter information is selected whose 
smoothing range describes a circular shape, and when the grade 
of the slope is large, filter information is selected whose 
smoothing range describes an elliptical shape. 

(3) The image processing method according to (1), wherein, 
at the filter information selection step, filter information 
is selected for which the direction of the slope corresponds 



to the inclination of the smoothing range. 



(4) The image processing method according to (1) further 
comprising: 

5 a smoothing strength calculation step of calculating 

luminance using the digital data, and calculating a smoothing 
U: strength using the luminance, 

i?l wherein, at the filter information selection step, filter 

Z] information is selected that corresponds to the edge information 

£i 10 obtained at the edge information calculation step and the 

m 

smoothing strength obtained at the smoothing strength 
M calculation step. 

CI (5) An image processing method, for smoothing digital data 

15 for an input image and removing noise included in the digital 
data, the method comprising: 

an edge information calculation step of extracting an 
edge of the image from the digital data, and calculating edge 
information that includes a grade and a direction of a slope 
2 0 of the edge; 

a filter information generation step of generating filter 
information based on the edge information obtained at the 
edge information calculation step; and 

a processing step of smoothing the digital data based 
25 on the filter information generated at the filter information 
generation step. 
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(6) A recording medium storing a computer program for causing 
an image processing apparatus, which includes an input unit 
for entering image data, a processor for processing digital 
data output by the input unit and a recording unit for recording 
filter information used to process the digital data, to smooth 
digital data for an input image and to remove noise included 
in the digital data, the image processing apparatus being 
caused by the computer program to perform: 

an edge information calculation step of extracting an 
edge of the image from the digital data received from the 
input unit, and calculating edge information including a grade 
and a direction of a slope of the edge; 

a filter information reading step of reading out specific 
filter information stored in the recording unit based on the 
edge information obtained at the edge information calculation 
step; and 

a processing step of smoothing the digital data based 
on the filter information read at the filter information reading 
step . 

(7) The recording medium according to (6) , wherein the filter 
information read at the filter information reading step is 
so set that when the grade of the slope is small, a smoothing 
range describes a circular shape, and when the grade of the 
slope is large, the smoothing range describes an elliptical 
shape . 



(8) The recording medium according to (6), wherein filter 
information read at the filter information reading step is 
so set that a inclination of the smoothing range corresponds 
to the direction of the slope. 

(9) The recording medium according to (6) , wherein the image 
processing apparatus is caused by the computer program to 
further perform: 

a smoothing strength calculation step of calculating 
luminance using the digital data, and of calculating a smoothing 
strength using the luminance, 

wherein, at the filter information reading step, specific 
filter information is read out from the recording unit based 
on the edge information obtained at the edge information 
calculation step and the smoothing strength obtained at the 
smoothing strength calculation step. 

(10) A recording medium storing a computer program for causing 
an image processing apparatus, which includes an input unit 
for entering image data and a processor for processing digital 
data output by the input unit, to smooth digital data for 
an input image and to remove noise included in the digital 
data, the image processing apparatus being causedby the computer 
program to perform: 

an edge information calculation step of extracting an 
edge of the image from the digital data, and calculating edge 
information that includes the grade and the direction of a 



slope of the edge; 

a filter information generation step of generating filter 
information based on the edge information obtained at the 
edge information calculation step; and 

a processing step of smoothing the digital data based 
on the filter information generated at the filter information 
generation step. 

(11) An image processing apparatus comprising: 

an image input unit for receiving image data and outputting 
the image data as digital data; 

an edge calculation unit for extracting an edge of an 
image from the digital data output by the image input unit, 
and for calculating edge information that includes the grade 
and the direction of a slope of the edge; 

a recording unit for storing filter information that is 
set in correlation with the edge information; 

a filter information selection unit for, based on the 
edge information calculatedby the edge information calculation 
unit, selecting filter information stored in the recording 
unit; and 

a smoothing unit for smoothing the digital data based 
on the filter information selected by the filter information 
selection unit. 

(12) An image processing apparatus according to (11), further 
comprising : 



a smoothing strength calculation unit for calculating 
luminance using the digital data, and calculating a smoothing 
strength using the luminance, 

wherein, on the recording unit, filter information is 
stored that is set in correlation with the edge information 
obtained by the edge information calculation unit and the 
smoothing strength obtained by the smoothing strength 
calculation unit. 

(13) An image processing apparatus comprising: 

an image input unit for receiving image data and outputting 

the image data as digital data; 

an edge calculation unit for extracting an edge of an 

image from the digital data output by the image input unit, 

and calculating edge information that includes the grade and 

the direction of a slope of the edge; 

a filter information generation unit for generating filter 

information stored in the recording unit based on the edge 

information calculated by the edge information calculation 

unit; 

a smoothing unit for smoothing the digital data based 
on the filter information generated by the filter information 
generation unit; and 

a recording unit for storing the smoothed digital data. 

According to (1), (6) or (11), digital data is smoothed 
using filter information corresponding to an edge. Further, 



since the filter information is set in advance and is selected 
based on the edge information that is obtained, a complicated 
conditional branching process need not be performed, and 
pipeline processing can be used to perform the smoothing process 
5 quickly. Therefore, the noise removal process can be performed 
rapidly without blurring the edges of images or emphasizing 
M the noise present in flat portions. 

Q According to (2) or (7), the filter information is so 

w 

%l set that as the slope of an edge is increased, the range for 

uii 

jfj 10 smoothing the filter information is changed from a circle 
to an ellipse. Therefore, when the slope of an edge is steep, 

m 

i.e., there is a great change in the luminance, the range 
\**4 for smoothing the filter information is changed to an elliptical 

shape, and the smoothing can be performed along the edge. 
15 Therefore, without emphasizing the noise present in a flat 

portion, along an edge, only the noise near the edge is removed. 
According to (3) or (8), the filter information is so 

set that the direction of a slope matches the inclination 

of the smoothing range. Thus, even when the shape of the edge 
20 is changed, the smoothing range can be changed in accordance 

with the shape of the edge. 

According to (4), (9) or (12), luminance is calculated 

based on the digital data, and the smoothing intensity is 

calculated using the luminance. The filter information is 
25 selected based on the smoothing intensity and the edge 

information. Thus, the smoothing intensity can be changed 

using the luminance of the edge, and noise near the edge can 
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be extensively removed. As a result, the noise near the edges 
can be removed without emphasizing the noise present in the 
flat portions. 

According to (5), (10) or (13), digital data is smoothed 
by using a filter that corresponds to an edge. The filter 
information is created based on the obtained edge information 
that is obtained. Therefore, complicated conditional 
branching processing need not be performed, and pipeline 
processing can be used to perform the smoothing processing 
quickly. Further, the filter information to be smoothed is 
changed in accordance with an image. And therefore, noise 
can be thoroughly removed in accordance with an image, without 
the edges of the image becoming blurred and noise present 
in flat portions being emphasized. 

BRIEF DESCRIPTION OF THE DRAWINGS 
Fig. 1 is a flowchart showing an image processing method 

according to a first embodiment of the present invention; 

Fig . 2 is a block diagram showing a digital camera according 

to the first embodiment of the present invention; 

Fig. 3 is a block diagram showing the processing performed 

by the digital camera according to the embodiment of the present 

invention; 

Fig. 4 is a specific diagram showing the CCD of the digital 
camera according to the first embodiment of the present 
invention; 

Fig. 5 is a specific diagram for explaining the edge 



- 10 - 



extraction process performed by the image processing method 
according to the first embodiment of the present invention; 

Fig. 6 is a diagram for explaining the relationship between 
the edge of a target pixel and the direction of the slope 
of the edge, for the image processing method according to 
the first embodiment of the present invention; 

Fig. 7 is a diagram for explaining the range for the 
smoothing of a target pixel for the image processing method 
according to the first embodiment of the present invention; 

Fig. 8 is a specific diagram showing a matrix depicting 
the filter information on the periphery of a target pixel 
according to the image processingmethod of the first embodiment 
of the present invention, and is used for explaining the 
relationship between a scaling parameter and a smoothing range; 

Fig. 9 is a diagram showing a matrix depicting the filter 
information on the periphery of the target pixel according 
to the first image processing method of the embodiment of 
the present invention, and is used for explaining the 
relationship between a smoothing intensity and a smoothing 
range employed for changing a scaling parameter; 

Fig. 10 is a specific diagram showing filter information 
when, for the image processing method according to the first 
embodiment, the smoothing intensity a = 0.9, the scaling 
parameter n = 4, m = 1.25 and the slope direction 9g = 45°; 

Fig. 11 is a diagram showing the processor of a digital 
camera according to a second embodiment of the present invention; 
and 



Fig. 12 is a flowchart showing the image processing method 
according to the second embodiment of the present invention. 
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DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 
A plurality of embodiments according to the present 
invention will now be described based on the drawings. 

First Embodiment 



Fig. 2 is a diagram showing a digital camera 1 for which 
jlj 10 an image processing apparatus according to a first embodiment 

m 

>i of the present invention is applied. 



As is shown in Fig. 2, the digital camera 1 comprises: 
a controller 10, an image input unit 20, a recording unit 
30, a display unit 40 and an interface 50. 

15 The controller 10 is an electric circuit for processing 

digital data output by the image input unit 20. The controller 
10 includes a CPU (Central Processing Unit) 11, a ROM (Read 
Only Memory) 12 and a processor 60. In the ROM 12, a computer 
program is recorded that is executed by the CPU 11 and the 

20 processor 60 of the controller 10. 

As is shown in Fig. 3, the processor 60 includes an edge 
information calculation unit 61, a filter information selection 
unit 63, a smoothing unit 64 and a writing unit 65. 

As is shown in Fig. 2, an input unit for receiving an 

25 entry from a user is connected to the controller 10. The input 
unit includes a shutter button 71 for receiving an imaging 
instruction from the user, and a plurality of input buttons 
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72 for receiving instructions for the performance of various 
functions of the digital camera 1. 

The image input unit 20 includes a condensing lens 21, 
a CCD 22 and anA/D converter 23 . The condensing lens 21 collects 
light from an object that is then transmitted to the CCD 22. 
The CCD 22 includes a plurality of imaging elements. These 
imaging elements are arranged horizontally and vertically 
in the shape of a matrix, and each imaging element constitutes 
one pixel. 

Color filters are arranged on the light receiving face 
of the imaging element. The color filters are complementary 
Cy (Cyan), Mg (Magenta), Ye (Yellow) and G (Green) filters. 
The complementary filters on the CCD 22 are arranged as shown 
in Fig. 4. Of the primary three light colors, i.e., red (R) , 
green (G) and blue (B) , the color filters pass light having 
the following colors: Cy = G + B, Mg = B + r, and Ye = G + 
R. That is, one filter passes light for two colors. 

The light entering the individual imaging elements of 
the CCD 22 is converted into an electric signal, and the electric 
signal is then output. Since the electric signal output by 
the CCD 22 is an analog signal, the A/D converter 23 converts 
the analog signal into digital data. 

The recording unit 30 includes a RAM (Random Access Memory) 

31 and a flash memory 32. A DRAM (Dynamic RAM) having a 
self-refresh function is used as the RAM 31. The flash memory 

32 is a writable recording medium on which stored contents 
can be retained even in the non-active state, and is either 



incorporated into the digital camera 1, or is detachable from 
the digital camera 1. 

The RAM 31 is used to temporarily store the digital data 
processed by the controller 10 or output from the A/D converter 
23. The flash memory 32 is used to accumulate and store the 
digital data that is temporarily stored in the RAM 31 . Further, 
filter information, which will be described later, is stored 
in the flash memory 32. 

The display unit 40 includes a liquid crystal display 
device (LCD) 41 and a VRAM (Video RAM) 42. The LCD 41 is used 
to display an image based on digital data stored in the flash 
memory 32 or output by the A/D converter 23. The VRAM 42 is 
used to store display data that is created from the digital 
data in order to display an image on the LCD 41. 

An interface 50 outputs digital data stored in the flash 
memory 32 to an external apparatus, such as a personal computer. 

The detailed processing performed by the controller 10 
will now be described. 

As previously described, a processor 60 that is provided 
for the controller 10 is a dedicated arithmetic unit for 
performing image processing. The processor 60 performs 
predetermined processing using a computer program stored in 
the ROM 32. 

The processing performed by means of the processor 60 
will be described by referring to Figs. 1 and 3. The pipeline 
processing is employed by means of the processor 60 without 
the CPU 11 using used. 
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Edge information calculation unit 

The edge information calculation unit 61 performs edge 
extraction processing for the digital data for an image output 
by the image input unit (S101) . An extracted edge is employed 
for calculations performed to determine the slope and the 
direction of the edge* 

For the extraction of the edge, a Prewitt operator is 
employed for obtaining the differential value of a pixel. 
The Prewitt operator is represented by the following equations . 
When the target pixel is defined as P(i, j), as is shown in 
Fig. 5, 

Afx = {P(i+1, j-1) - P(i-1, j-1) } + {P(i+1, j) - P(i-1, 
j) } + {P(i+1, j + 1) - P(M, j+1) } 

Afy = {P{H, j + 1) - P(i-1, j-1)} + {P(i, j + 1) - P(i, 
j-1)} + {P(i+1, j + 1) - P(i+1, 

In these equations, edge extraction is performed based 
on the digital data that is output from pixels neighboring 
the target pixel which constitute a matrix 80 (3x3 matrix 
around the target pixel P in this embodiment) and include 
the target pixel P in the center as shown in Fig. 5 and include 
the target pixel P in the center. During the edge extraction 
process, the edge elements in the directions X and Y, indicated 
by arrows, are detected. When the output is 8 bits, 256 tone 
data levels, 0 through 255, are output by the pixels as digital 
data. 

Next, edge information, including the slope and the 
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direction of the edge, is calculated using the edge extracted 
by the equations (S102) . The grade grad of the slope of the 
edge is calculated using the following equation (A) , and the 
direction Gg of the slope of the edge is calculated using 
the following equation (B) . 



G(q>) = J/Lfl + Xf y ....(A) 
A/ 



& g = tan 



....(B) 

10 



Using the equations (A) 
and (B) , the edge information calculation unit 61 calculates, 
as edge information, the grade grad and the direction 0g of 
the slope of the edge. 

Further, instead of equation (A), equation (Al) or (A2) 
may be employed to calculate the grade grad of the slope of 
the edge. When the equation (Al) or (A2 ) is employed, the 
grade grad of the slope of the edge can be quickly obtained. 

grad(cp) = |Af x | + |Af y | .... (Al ) 

grad(cp) = max(Af x , Af y ) .... (A2) 

The edge is a portion whereat the tone of the digital 
data output by the pixel is greatly changed, i.e., the boundary 
portion between the objects included in the image. In this 
embodiment, the tone of the digital data output by the pixel 
is greatly changed on both sides of the edge. The change in 
the tone of the digital data is defined as the grade grad 
of the slope, and the direction in which the tone of the digital 



data is changed is defined as the direction 8g of the slope. 

Therefore, as is shown in Fig. 6, the direction of the edge 
E is perpendicular to the direction Gg of the slope of the 
edge . 

Smoothing strength calculation unit 

Not only the edge information, but also the smoothing 
strength is calculated using the digital data output by the 
image input unit 20 (S103) . 

The smoothing strength calculation unit 62 calculates 
smoothing strength o based on the information for luminance 
Y included in the digital data that is output by each pixel 
of the CCD 22, The smoothing strength a is obtained by solving 
the following equation (C) : 



a = N(Y) = 




e 



denotes a natural 
logarithmic number, 

and n denotes a luminance at which the noise reaches the maximum. 

The luminance n is changed in accordance with the imaging 
condition, and in this embodiment, n = 50 when the output 
by the pixel is 8 bits, 256 tones. 

The smoothing range is calculated using the smoothing 
strength a obtained by (C) . The Gaussian distribution, which 
will be described later, is employed to obtain the smoothing 
range . 



Filter information selection unit 

Based on the edge information obtained by the edge 
information calculation unit 61 and the smoothing strength, 
the filter information selection unit 63 selects filter 
5 information stored in the flash memory 63 . From among multiple 
sets of stored filter information, the filter information 
*fc selection unit 63 selects the filter information that 

If corresponds to the obtained edge information and the smoothing 

y strength (S104) . 

jl{ io The filter information is the one obtained by, based on 

the edge information- modifying a smoothing filter that is 
set using the Gaussian distribution. The principle will now 
be described by using an example. In this embodiment, a 5 

: =? Si 

CI x 5 matrix, including the target pixel P in the center, is 

rll 

15 defined as a smoothing range. 

The Gaussian distribution is obtained by solving the 
following equation (D) : 



CI 



J_ 
2n cf 2 



f n (x cos d g - y sin 0 g f + m (x sin 0 g + y cos $ g f 



Equat 
ion (D) 
represents 

a Gaussian distribution that is modified to describe an ellipse 
25 by scaling the short axial direction using 1/n and the long 
axial direction using 1/m, and is rotated to the right in 
a direction 6g. That is, the elliptical shape is changed by 



- 18 - 



m and n and the inclination of the ellipse is changed by 0g. 
The m and n values are scaling parameters and depend on the 
grade grad of the slope obtained by equation (A) . 

As the grade grad of the slope of the edge is increased, 
5 the values of the scaling parameters m and n (mainly the value 
of n) are increased, and the Gaussian distribution shape is 

[3!??: 

i«H changed and describes a flat elliptical shape. Further, as 

;|J the ellipse is rotated in accordance with the slope direction 

Jjj 9g, the smoothing range A along the edge E is obtained, as 

[It 10 is shown in Fig. 7. That is, when the values of the scaling 
'{^ parameters m and n are small, the smoothing range Al describes 

I as J 

'41 a circle, as indicated by the broken line in Fig. 7. On the 

iJI other hand, as the scaling parameters m and n, especially 

m the value of n, are increased, the smoothing range A2 assumes 

15 a flat elliptical shape, as is indicated by the solid line 
in Fig. 7. 

As a result of the calculation of the edge information 
for the target pixel P, it is found that the periphery of 
the target pixel P is a flat portion when the slope of the 
20 edge is small. Therefore, the digital data output by the target 
pixel P must be equally distributed throughout the peripheral 
pixels of the target pixel P. Thus, the smoothing range is 
shaped like a circle that includes the target pixel P in the 
center . 

25 When the slope of the edge is large, the periphery of 

the target pixel P is an edge portion. Therefore, the digital 
data output by the target pixel P must be distributed throughout 
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the peripheral pixels of the target pixel P along the edge. 

Thus, the smoothing range is shaped like an ellipse along 
the edge E along the target pixel P in the center. That is, 
as the slope of the edge is increased, the smoothing process 
5 must be performed along the edge E. 

In this embodiment, the grade grad of the slope of the 
53 edge obtained by the edge information calculation unit 61 

ft is correlated with the values of the scaling parameters m 

%l 

i|| and n. Thus, when the grade grad of the slope is determined, 

ill 

\% 10 scaling parameters m and n are determined that correspond 

fn to the grade grad of the slope. 

iVi 

% Fig. 8 is a diagram showing the change in the smoothing 

: si; l 

H! range when the scaling parameter n is changed while the direction 

W eg = 45° of the slope and the smoothing strength a = 0.7 are 

15 maintained as an example. The matrix is not limited to the 
5x5 dimension used in this embodiment, and can be an 
appropriately set dimension, such as 3 x 3 or 7 x 7 . 

It is apparent from Fig. 8 that when n = 1 the smoothing 
range is uniform in all directions, and that as the value 
20 of n is increased, the axis along the direction 9g of the 
slope is shortened and the smoothing range is concentrated 
along the edge E. Further, when the scaling parameter n is 
greater than n = 3, the smoothing range is substantially 
unchanged. Therefore, when a = 0.7, the effective range of 
25 the scaling parameter n is from 1 to 3, 

An explanation will now be given for the relationship 
between the smoothing strength and the scaling parameter. 
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The smoothing range can be changed in accordance with the 
level of the smoothing strength a using the equation (D) . 
When the smoothing strength a is small, the smoothing effect 
is small, and when the smoothing strength a is large, the 
5 smoothing effect is great. 

Since the smoothing range is increased as the smoothing 
U strength a becomes greater, the smoothing range must be reduced 

Q by increasing the scaling parameter n. Further, since the 

Sj 

ij effects of smoothing the edge become too great if only the 

: s 

5| 10 scaling parameter n in the shorter axial direction is changed, 
the scaling parameter m in the longer axial direction must 
also be changed. 

Fig. 9 is a diagram showing filter information that provides 
the minimum smoothing effect and filter information that 
15 provides the maximum smoothing effect, which are obtained 
in case of changing the smoothing strength a. The left column 
in Fig. 9 represents the filter information that provides 
the minimum smoothing effect, and the right column represents 
the filter information that provides the maximum smoothing 
20 effect. 

Multiple filter information sets that correspond to the 
individual parameters are stored in the flash memory 32. These 
filter information sets include a smoothing range that is 
set by combining the scaling parameters m and n and the direction 
25 9g of the slope, which are determined by the grade grad of 
the slope, and the smoothing strength a that is calculated 
using the luminance Y. 
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The filter information selection unit 63 reads out specific 
filter information from multiple filter information sets that 
are stored in the flash memory 32. That is, the values of 
the scaling parameters m and n are determined from the grade 
5 grad of the slope of the edge that is calculated by the edge 
information calculation unit 61. Then, based on the scaling 
Cl parameters m and n that are thus determined, the direction 

9S 9g of the slope, and the smoothing strength a obtained by 

si 

; !S the smoothing strength calculation unit 62, the specific filter 

m 

|J| 10 information is selected and is read out from the flash memory 
h 32 . 

hi 



m 
ci 



Smoothing unit 

When the filter information is selected by the filter 

15 information selection unit 63, the smoothing process is 
performed by using the digital data output by the target pixel 
P and the selected filter information (S105) . The smoothing 
process is performed by multiplying the row data of the digital 
data output by the target pixel P by the numerical value of 

20 the filter information. 

Assume that the scaling parameters m = 1.25 and n = 4 
are determined from the grade grad of the slope obtained by 
the peripheral matrix 80 of the target pixel P, the inclination 
of the slope is 0g = 45°, and the smoothing strength o = 0.9 

25 is obtained based on the luminance Y of the target pixel P. 
Then, filter information 90 shown in Fig. 10 is selected by 
the filter information selection unit 63. When the row data 
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of the digital data output by the target pixel P is multiplied 
by the numerical value (the numerical value of a matrix/the 
sum) of the filter information in Fig. 8, the information 
for the target pixel P is distributed among specific pixels 
5 of the 5x5 matrix 80, including the target pixel P in the 
center. Since the information for the target pixel P is 

£J distributed among the specific pixels of the matrix 80, the 

smoothing process is performed for the digital data for the 

% i target pixel P. 

41 

111 10 This smoothing process is performed for all the pixels 

ii that constitute the image, i.e., all the pixels in the CCD 

Ul 22 • Since the smoothing process is performed for the 2 4 

ijf peripheral pixels of onepixel, the smoothingprocess is repeated 

I» 25 times for one pixel. The total of the data obtained bv 

15 the smoothing process performed for the peripheral pixels 
becomes the digital data obtained by the smoothing. 

Writing unit 

When the smoothing process is completed by the smoothing 
20 unit 64, the obtained digital data is written to the RAM 31 
by the writing unit 65. 

When the digital data for one image is written to the 
RAM 31 by the writing unit 65, the digital data stored in 
the RAM 31 is compressed in order to reduce the amount of 
25 data that is to be stored in the flash memory 32. When data 
for an image obtained by the digital camera 1 is to be compressed, 
a file form, such as JPEG (Joint Photographic Experts Group) 
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or TIFF (Tagged Image File Format) , is employed. The compressed 
digital data is recorded in the flash memory 32. 

Operation of digital camera 

The operation performed by the digital camera 1 according 
to the first embodiment will now be described. 

(1) When the power switch (not shown) of the digital camera 
1 is turned on, the digital camera 1 is set to the standby 
state so that the image pickup is always ready. At this time, 
the CCD 22 converts into an electric signal light that is 
condensed by the condenser 21 every tenth to several hundredths 
of a second. The obtained electric signal is then converted 
to digital data by the A/D converter 23. When a user employs 
the LCD 41 as a viewfinder, the digital data output by the 
A/D converter is transmitted to the VRAM 42, and the object 
is displayed on the LCD 41 as an animated picture. 

(2) When the user depresses the shutter button 71 to the 
middle of the operating range and the shutter button 71 is 
set to the "half-depressed" state, the exposure and focus 
are set and fixed. The exposure fixed during the imaging can 
be changed when the CPU 11 of the controller 10 controls the 
aperture of the condensing lens 21 or the shutter speed, i.e., 
the charge accumulation period for the CCD 22. One of or both 
of a mechanical shutter for physically shielding light and 
a digital shutter for controlling the charge accumulation 
time for the CCD 22 are employed as the shutter of the digital 
camera 1 . 



(3) When the shutter button 71 is set to the "fully depressed 
state" in which the user depresses the button 71 to the limit 
of the operating range, the following process is performed* 
First, light measurement and focusing are precisely performed 

5 for the object. When the light measurement and the focusing 
have been completed, all the charges accumulated in the CCD 
M; 22 are temporarily discharged, and light from the object passes 

0 through the condenser 21 and enters the CCD 22. The CCD 22 

m 

%l then outputs an electric signal based on the strength of charges 

it. 

fll 10 that correspond to the amount of incident light. 

(4) The electric signal output by the CCD 22 is then 

gjj converted into digital data by the A/D converter 23. And in 

It 

fit order to increase the processing speed, DMA (Direct Memory 

'5? » 

1 sic;- 

5; Access) is employed to directly, temporarily store the digital 

15 data at a designated address in the RAM 31 without the data 
passing through the CPU 11 of the controller 10. 

(5) The processor GO performs the above processing for 
the digital data stored in the RAM 31, and regards the resultant 
data as digital data for an appropriate color image. In order 

20 to increase the number of images recorded in the flash memory 
32, the obtained data is compressed to obtain digital data 
having a JPEG file form. 

(6) After the digital data has been compressed, the digital 
data is copied from the RAM 31 to the flash memory 32 and 

25 is recorded therein. 

As is described above, according to the image processing 
method performed by the digital camera 1 according to the 
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first embodiment of the present invention, filter information 
is selected and read based on the scaling parameters n and 
m, which are determined by the grade grad of the slope of 
the edge that passes through the target pixel P, the direction 
5 6g of the slope of the edge, and the smoothing strength a 
calculated using the digital data output by the target pixel 
M P> Therefore, appropriate filter information corresponding 

CI to the edge can be selected without complicated conditional 

m 

H branching being required. In addition, the filter information 

i|i 

Fj 10 is designated and stored in the flash memory 32, i.e., filter 
7 information appropriate for processing digital data output 

JJ by the target pixel P is selected based on the obtained edge 

U information and the smoothing strength a. Therefore, the 

Iff smoothing can be performed quickly using pipeline processing. 

15 Thus, noise can be removed quickly, without the edge of 

an image being blurred or the noise present in a flat portion 
being emphasized. 

Furthermore, the information for the smoothing range 
included in the filter information is so set that as the grade 
20 grad of the slope of the edge is increased, the smoothing 
range is changed from a circular into an elliptical shape. 

Thus, noise near the edge can be thoroughly removed without 
emphasizing noise present in the flat portion. 

Further, when the filter information is selected using 
25 the edge information and the smoothing strength, a function 
for smoothing the data in accordance with the slope of the 
edge and for emphasizing the contour and a function for changing 
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the smoothing range can be performed at the same time. Thus, 
a fast noise removal process can be performed. 



Second Embodiment 
5 A processor 100 for a digital camera according to a second 

. . embodiment of the present invention will now be described 

Cl while referring to Figs. 10 and 11. No explanation will be 

ff 

81 given for components and processing that are substantially 

i£| the same as those in the first embodiment. 

FfJ 10 The second embodiment differs from the first embodiment 

in that filter information is generated each time the grade 

U-I 

m grad of the slope of the edge and the direction 9g of the 

ijf 

m slope, both of which constitute the edge information. 

As is shown in Fig. 11, the processor 100 for the digital 

15 camera in the second embodiment comprises: an edge information 
calculation unit 101, a smoothing strength calculation unit 
102, a filter information generation unit 103, a smoothing 
unit 104 and writing unit 105. 

The edge information calculation unit 101 and the smoothing 

20 strength calculation unit 102 are the same as those in the 
first embodiment. The filter information generation unit 103 
generates filter information using equation (D) , as explained 
in the first embodiment, based on the edge information obtained 
by the edge information calculation unit 101 and the smoothing 

25 strength obtained by the smoothing strength calculation unit 
102. That is, each time the edge information and the smoothing 
strength are calculated based on the digital data output by 
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the pixels that constitute an image, filter information is 
generated for the individual pixels. 

In the second embodiment, theprocess (S201) for extracting 
an edge from the digital data output by the CCD 22, the process 
5 (S202) for calculating the edge information using the extracted 
edge and the process (S203) for calculating the smoothing 
h strength are the same as those as in the first embodiment. 

If In the second embodiment, the scaling parameters m and n are 

SI set based on the edge information and the smoothing strength 

m 10 obtained by the filter information generation unit 103. The 
filter information generation unit 103 generates filter 
H information using equation (D) (S204) . The smoothing unit 

S 104 employs the filter information obtained by the filter 

y i 

S! information generation unit 103 to smooth the digital data 

' lb 

15 output by the pixel (S205) . And the smoothed digital data 
is stored by the writing unit 105 in the RAM 31 of the recording 
unit 30. 

In the second embodiment, each time the edge information 
and the smoothing strength are calculated using the digital 

20 data output by the pixel, the scaling parameters m and n are 
designated, and filter information corresponding to each pixel 
is generated. Therefore, noise can be effectively removed 
without the edge being blurred. 

In these embodiments of the invention, the luminance Y 

25 has been calculated after the edge information has been is 
obtained. However, the luminance Y may be calculated first, 
or the edge information and the luminance Y may be calculated 
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in parallel, at the same time* Further, the smoothing range 
may be determined not only based on the luminance Y, but also 
based on the strength of the color element, such as RGB. 

Furthermore, in the embodiments of the invention, the 
5 smoothing range is changed by altering the smoothing strength 
and the scaling parameters m and n. However, even when the 
CI smoothing strength <p is fixed, the smoothing range can be 

iS] changed by changing the scaling parameters m and n . Therefore, 

%l ^ - ^ . 

;|l when the value of the smoothing strength cp is fixed at a constant 

iff 

1^1 10 value, and the scaling parameters m and n are changed based 

;L on the grade and the direction of the slope of the edge, the 

! 51 

J?f filter information can more quickly be selected or generated. 

■i! Further, in the embodiments of the invention, the Prewitt 

s 

ril operator is employed as a method for extracting the grade 

15 and the direction of the slope of the edge. However, another 
edge extraction method, such as the Sobel operator or the 
Kirsch operator, may be employed. 

In addition, in the embodiments of the invention, a color 
image of 256 tones for which a complementary filter is used 
20 has been employed as an image to be input. However, a color 
image using the RGB primary color filter, a color image having 
1024 or 4096 tones, a gray scale image or a binary monotone 
image may be employed as an input image. 

Moreover, in the embodiments of the invention, an 
25 explanation is given for the processing of the digital data 
output by a CCD having a complementary filter. However, not 
only a complementary filter, but also a RGB primary filter 
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can be employed. 

In the embodiments of the invention, a digital camera 
is employed as an image processing apparatus. However, the 
present invention can be applied for image processing performed 
not only by a digital camera, but also by an image reading 
apparatus, such as a scanner or a copier, or for image processing 
for which software, such as a printer driver, is used. 



